Papers with ERASER benchmark
AdapLeR: Speeding up Inference by Adaptive Length Reduction (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have shown stellar performance in downstream tasks, but their excessive computational costs and high latency hinder their usage in resource-limited settings. |
| Approach: | They propose a method that dynamically eliminates less contributing tokens through layers, resulting in shorter lengths and consequently lower computational cost. |
| Outcome: | The proposed method shows speedups up to 22x during inference time without much sacrifice in performance. |
Make Your Decision Convincing! A Unified Two-Stage Framework: Self-Attribution and Decision-Making (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing frameworks for explaining black-box model behavior are unreliable . large-scale pre-trained models often rely on superficial clues for predictions . |
| Approach: | They propose a unified two-stage framework that uses subsequences from the input text as a rationale to generate model decision. |
| Outcome: | The proposed framework achieves competitive results on five reasoning datasets and in semi-supervised scenarios. |
An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to condition models on a concise rationale are less accurate than models that can use the entire context. |
| Approach: | They propose a method to optimize a bound on the Information Bottleneck objective to extract concise rationales from a binary mask and an end-task predictor that uses only the residual sentences. |
| Outcome: | The proposed model outperforms existing norm-minimization techniques in task performance and agreement with human rationales in the ERASER benchmark. |
Rational LAMOL: A Rationale-based Lifelong Learning Framework (2021.acl-long)
Copied to clipboard
Kasidis Kanwatchara, Thanapapas Horsuwan, Piyawat Lertvittayakumjorn, Boonserm Kijsirikul, Peerapon Vateekul
| Challenge: | Existing paradigms for machine learning suffer from catastrophic forgetting when a model completely forgets what it just learned in previous tasks. |
| Approach: | They propose to exploit unsupervised rationale generation to improve the performance of a lifelong language learning model by applying critical freezing guided by human rationales. |
| Outcome: | The proposed framework outperforms vanilla LAMOL on most permutations and unsupervised rationale generation consistently improves the overall performance. |